build(deps): update bitsandbytes requirement from <1,>=0.45 to >=0.50.0,<1 in /infra/ai-server - #50

Sourced from bitsandbytes's releases.

0.50.0: Faster 4-bit inference, CPU/ROCm/Apple Silicon improvements, and more!

This release brings a new fused 4-bit GEMM for inference on CUDA and ROCm, faster CPU ops on x86-64 and ARM64, reduced host-side overhead, and a much improved Apple Silicon backend. We've also added Windows on ARM CPU support, ROCm builds for Windows, additional ROCm and CUDA build variants, and new optimizer support on CPU and Intel XPU.

Highlights

🚀 New 4-bit GEMM kernels for inference (CUDA)

New fused 4-bit dequantize + GEMM kernels replace the old GEMV and dequantize + F.linear paths for small-to-medium batch sizes. 4-bit inference is up to 4x faster at batch sizes of 2 through 64 across Turing through Blackwell, with wins at batch size 1 in many cases too. Nested (double) quantization and bias are fused in as well, so nested quant sees an additional benefit. Kernel selection happens automatically at runtime based on shape, GPU architecture, and SM count. See #1949 for benchmarks and details.

🚀 Fused 4-bit SIMT GEMM on ROCm

The SIMT version of the new 4-bit GEMM has been ported to ROCm and wired into the same dispatch for small inference batches, validated on gfx1100, gfx1201, and gfx1151 (#1979).

AMD ROCm improvements

Stability and performance improvements bring AMD ROCm support out of preview; it is now considered stable.

• ROCm wheels for Windows are now published for ROCm 7.2 and 7.14 (#1915, #2007).
• Added ROCm 7.14 and dropped the ROCm 6.2 and 6.3 builds (#2007).
• Expanded GPU target coverage to add RDNA2 consumer GPUs, additional RDNA3 and RDNA3.5 GPUs, and CDNA1 (gfx908) data center GPUs (#1886, #2007).

Apple Silicon (MPS)

Apple Silicon support is improved. The MPS backend added optimized Metal kernels from the Hub (#1875) and was substantially improved (#1960, #1983, #1994) so that all 4-bit and LLM.int8() configurations now work on MPS. On macOS 26+, install the kernels package for the best performance, which enables the optimized Metal kernels; otherwise a naive fallback is used. The MPS path requires torch >= 2.9.

The last remaining feature for support parity is the 8bit optimizers, which will land in a future release.

CPU performance on x86-64 and ARM64

Blockwise quantization and dequantization on CPU are considerably faster, mostly from better SIMD usage plus some compile flag tuning. Improvements range from 1.1x to over 20x depending on op, dtype, and hardware, with the largest gains on fp16 and on x86-64 CPUs without AVX-512. The LLM.int8() matmul on CPU was improved as well. See #1968 for benchmarks.

Reduced host-side CPU overhead

Reduced Python dispatch overhead, especially on the CUDA/ROCm backend (#1953).

Windows on ARM CPU support

Windows ARM64 CPU wheels are now built with NEON-optimized kernels (#1959), with nightly test coverage added (#1962).

New optimizer support

• CPU optimizers are now supported for both 32-bit and 8-bit blockwise (#1901).
• Intel XPU gained paged optimizer support (#1898) and support for the LARS and LAMB optimizers (#1902).
• Fixed Lion to use decoupled weight decay (#1993, #2001).
• Other optimizer fixes (#1869, #1998, #1855).

NVIDIA CUDA

• Added CUDA 13.2 build wheels and nightly test coverage (#1925).
• Smaller wheels: reduced the CUDA build matrix and added a closest-version fallback for library loading (#1980).

Intel XPU

• Added a oneAPI 2026 build; wheels now package both the 2025 and 2026 variants and pick the right one at runtime based on the PyTorch XPU version (#2002).
• Fixed gemv_4bit bf16 correctness on Intel Arc A-series (Alchemist) GPUs (#1942).

🚨 Breaking Changes

• Minimum PyTorch version is now 2.4 (#1926).
• Removed deprecated APIs: the research module, non-blockwise (block_wise=False) optimizers, and legacy dynamic quantization functions, along with their CUDA/HIP kernels (#1871, #1880).
• Removed legacy sparse functionality (spmm_coo, spmm_coo_very_sparse) and dropped the cusparse/hipsparse dependencies (#1881).

... (truncated)

Sourced from bitsandbytes's changelog.

v0.45.1

Improvements:

• Compatibility for triton>=3.2.0
• Moved package configuration to pyproject.toml
• Build system: initial support for NVIDIA Blackwell B100 GPUs, RTX 50 Blackwell series GPUs and Jetson Thor Blackwell.
  • Note: Binaries built for these platforms are not included in this release. They will be included in future releases upon the availability of the upcoming CUDA Toolkit 12.7 and 12.8.

Bug Fixes:

• Packaging: wheels will no longer include unit tests. (#1478)

Dependencies:

• Sets the minimum PyTorch version to 2.0.0.

0.45.0

This is a significant release, bringing support for LLM.int8() to NVIDIA Hopper GPUs such as the H100.

As part of the compatibility enhancements, we've rebuilt much of the LLM.int8() code in order to simplify for future compatibility and maintenance. We no longer use the col32 or architecture-specific tensor layout formats while maintaining backwards compatibility. We additionally bring performance improvements targeted for inference scenarios.

Performance Improvements

This release includes broad performance improvements for a wide variety of inference scenarios. See this X thread for a detailed explanation.

Breaking Changes

🤗PEFT users wishing to merge adapters with 8-bit weights will need to upgrade to peft>=0.14.0.

Packaging Improvements

• The size of our wheel has been reduced by ~43.5% from 122.4 MB to 69.1 MB! This results in an on-disk size decrease from ~396MB to ~224MB.
• Binaries built with CUDA Toolkit 12.6.2 are now included in the PyPI distribution.
• The CUDA 12.5.0 build has been updated to CUDA Toolkit 12.5.1.

Deprecations

• A number of public API functions have been marked for deprecation and will emit FutureWarning when used. These functions will become unavailable in future releases. This should have minimal impact on most end-users.
• The k-bit quantization features are deprecated in favor of blockwise quantization. For all optimizers, using block_wise=False is not recommended and support will be removed in a future release.
• As part of the refactoring process, we've implemented many new 8bit operations. These operations no longer use specialized data layouts.

Full Changelog

• refine docs for multi-backend alpha release by @​Titus-von-Koeller in #1380
• README: Replace special Unicode text symbols with regular characters by @​akx in #1385
• Update CI tools & fix typos by @​akx in #1386
• Fix invalid escape sequence warning in Python 3.12 by @​oshiteku in #1420
• [Build] Add CUDA 12.6.2 build; update 12.5.0 to 12.5.1 by @​matthewdouglas in #1431
• LLM.int8() Refactoring: Part 1 by @​matthewdouglas in #1401

0.44.1

Bug fixes:

... (truncated)

• c5d481b ci: skip-existing on PyPI publish
• ba9bc38 Release 0.50.0
• f54075e Update supported platforms in installation guide (#2019)
• fbbfc09 Update README (#2017)
• c23a72e (docs) update XPU section (#2016)
• 7d1f6a2 docs: update AMD section (#2015)
• 6721333 chore(deps): bump softprops/action-gh-release in the actions group (#2013)
• dde003b fix(rocm): avoid SIMT workgroup-count resonance (#2012)
• f2233a6 fix(optim): honor user-provided max_unorm in LAMB (+ 8-bit arg guards) (#1998)
• a229e65 cleanup: remove unused CUDA install scripts (#2009)
• Additional commits viewable in compare view